A Hierarchical Framework of Cloud Resource Allocation and Power Management Using Deep Reinforcement Learning

Automatic decision-making approaches, such as reinforcement learning (RL), have been applied to (partially) solve the resource allocation problem adaptively in the cloud computing system. However, a complete cloud resource allocation framework exhibits high dimensions in state and action spaces, which prohibit the usefulness of traditional RL techniques. In addition, high power consumption has become one of the critical concerns in design and control of cloud computing systems, which degrades system reliability and increases cooling cost. An effective dynamic power management (DPM) policy should minimize power consumption while maintaining performance degradation within an acceptable level. Thus, a joint virtual machine (VM) resource allocation and power management framework is critical to the overall cloud computing system. Moreover, novel solution framework is necessary to address the even higher dimensions in state and action spaces. In this paper, we propose a novel hierarchical framework for solving the overall resource allocation and power management problem in cloud computing systems. The proposed hierarchical framework comprises a global tier for VM resource allocation to the servers and a local tier for distributed power management of local servers. The emerging deep reinforcement learning (DRL) technique, which can deal with complicated control problems with large state space, is adopted to solve the global tier problem. Furthermore, an autoencoder and a novel weight sharing structure are adopted to handle the high-dimensional state space and accelerate the convergence speed. On the other hand, the local tier of distributed server power managements comprises an LSTM based workload predictor and a model-free RL based power manager, operating in a distributed manner.Comment: accepted by 37th IEEE International Conference on Distributed Computing (ICDCS 2017

Survivable Virtual Infrastructure Mapping in Virtualized Data Centers

In a virtualized data center, survivability can be enhanced by creating redundant VMs as backup for VMs such that after VM or server failures, affected services can be quickly switched over to backup VMs. To enable flexible and efficient resource management, we propose to use a service-aware approach in which multiple correlated Virtual Machines (VMs) and their backups are grouped together to form a Survivable Virtual Infrastructure (SVI) for a service or a tenant. A fundamental problem in such a system is to determine how to map each SVI to a physical data center network such that operational costs are minimized subject to the constraints that each VM’s resource requirements are met and bandwidth demands between VMs can be guaranteed before and after failures. This problem can be naturally divided into two sub-problems: VM Placement (VMP) and Virtual Link Mapping (VLM). We present a general optimization framework for this mapping problem. Then we present an efficient algorithm for the VMP subproblem as well as a polynomial-time algorithm that optimally solves the VLM subproblem, which can be used as subroutines in the framework. We also present an effective heuristic algorithm that jointly solves the two subproblems. It has been shown by extensive simulation results based on the real VM data traces collected from the green data center at Syracuse University that compared with the First Fit Descending (FFD) and single shortest path based baseline algorithm, both our VMP+VLM algorithm and joint algorithm significantly reduce the reserved bandwidth, and yield comparable results in terms of the number of active servers

Enhancing survivability and reliability in the cloud

Cloud computing has evolved as an important distributed computing model, enabling infrastructure, information, and software to be used as shared resources over the network in an on-demand manner. A complex cloud computing system is a large-scale network which may include multiple Data Centers (DCs) that are distributed all over the world. Given a large number of connected servers across the whole planet, server and network failures are inevitable. Meanwhile, the power-demanding nature of data centers urges us to allocate existing resources efficiently, instead of simply adding more servers, to enhance the reliability. It is meaningful to study how to build survivable and reliable cloud computing systems while maximizing power efficiency. In this dissertation, we first studied survivable Virtual Machine (VM) management by proposing a general optimization framework, designing a polynomial-time optimal algorithm and an efficient heuristic algorithm for virtual link mapping and VM placement subproblems respectively, as well as designing an effective algorithm to solve the two subproblems jointly. We reduced the reserved bandwidth by at least 96% and yielded comparable results in terms of the number of active servers against the first fit decreasing and single shortest path-based baseline algorithm in the simulation. We also studied the reliable VM management problem and proposed the first Deep Reinforcement Learning(DRL)-based continuous-time and event-driven resource allocation framework that combines a deep neural network with autoencoder and novel weight sharing structure, and an online deep Q-learning framework to handle high-dimensional state space. Simulation results showed that our DRL-based solution reduced the server power consumption by least 47% with minor job latency increases in generating reliable VM placement, compared with the round-robin baseline algorithm. Moreover, we studied reliable VM management in distributed DCs by formulating Virtual Server Provisioning and Selection (VSPS) as a mixed integer linear programming problem, and proposing a novel optimization framework, under which we developed a polynomial-time ln(N)-approximation algorithm, along with a heuristic algorithm which jointly solves sub-problems of the VSPS. Simulation results showed that our algorithms provide close-to-optimal performance and achieve 25% or more cost reduction compared to a baseline algorithm. Finally, we studied the reliability enhancement for Distributed Stream Data Processing Systems (DSDPSs) and designed a predictive DSDPS control framework which consists a two-tiered Deep Recurrent Neural Network (DRNN) model with consideration on co-location interference and implementing dynamic grouping for misbehaving workers bypassing. In addition, we implemented and tested our framework over a well-known DSDPS Storm and showed that our DRNN model outperformed AutoRegressive Integrated Moving Average(ARIMA) and Support Vector Regression (SVR) in terms of prediction accuracy, and that our framework introduced minor performance degradation when misbehaving workers exist

Free-standing microporous paper-like graphene films with electrodeposited PPy coatings as electrodes for supercapacitors

Free-standing microporous paper-like graphene films with electrodeposited polypyrrole (PPy) coatings were prepared. The microporous structures were produced by employing PS microspheres as sacrificial templates. PPy was coated on the films using an electrochemical deposition process to further improve the performance of these graphene electrode materials. The electrochemical performance of PPy coated microporous graphene films is evaluated and compared with solid graphene films. The results reveal that the incorporation of PPy and microporous structures significantly improve the electrochemical performance of graphene based electrodes for supercapacitors. Microporous films have higher capacitance than their solid counterparts although they have slightly lower conductivity, which reveals that microporous structures play an important role in the electrochemical performance of the electrode materials. The PPy coated microporous graphene films have the better electrochemical performance than uncoated counterparts, which indicates that PPy coatings have significant effects on graphene based electrodes. This in-depth research on free-standing microporous paper-like graphene with electrodeposited PPy coatings provides a new route to combine the advantages of both graphene and PPy so as to produce high performance electrodes for supercapacitors

Overexpression of CENPL mRNA potentially regulated by miR-340-3p predicts the prognosis of pancreatic cancer patients

Abstract Background In our previous study it was found that CENPL was overexpressed in hepatocellular carcinoma and significantly predicted patient's prognosis. However, the expression and prognostic value of CENPL in other gastrointestinal tumors remain unknown. Therefore, we investigated the expression and prognostic value of CENPL in esophageal carcinoma (ESCA), stomach adenocarcinoma (STAD), pancreatic adenocarcinoma (PAAD), colon adenocarcinoma (COAD) and rectum adenocarcinoma (READ). Methods In this study, Oncomine, GEPIA, OncoLnc, TIMER, cBioPortal, miRWalk and ENCORI databases were used to analyze the level of CENPL mRNA, prognostic value and potential regulatory mechanism of CENPL mRNA in tumors. The CENPL expression and clinicopathological data regarding PAAD were from the UCSC Xena database and univariate and multivariate Cox regression analyses were performed using R (Version 3.6.3). Immunohistochemical staining was used to verify the expression of CENPL protein in clinical specimens. Cytoscape (Version: 3.7.2) was used to visualize microRNA (miRNA) that potentially regulates CENPL. Results Gene differential expression analysis showed that CENPL mRNA was significantly overexpressed in ESCA, STAD, PAAD, COAD and READ (p  0.05). Univariate and multivariate Cox regression analyses suggested that CENPL was a prognostic risk factor for PAAD. The mutation rate of CENPL in PAAD was 2.2% (17/850). There was no significant correlation between the CENPL expression and the infiltration levels of immune cells in PAAD (|Cor|< 0.5). Immunohistochemical staining showed that CENPL was overexpressed in 42% (11/26) of PAAD specimens, which was significantly higher compared with that in the normal tissues. The expression of miR-340-3p and miR-484 in PAAD were significantly lower than in the normal tissues (p < 0.05) and PAAD patients with lower expression of miR-340-3p had poorer prognosis (p < 0.05). Conclusion CENPL potentially regulated by miR-340-3p, is overexpressed in PAAD and predicts patient’s prognosis, suggestive of a diagnostic and prognostic value in PAAD patients

Bioadhesive Microporous Architectures by Self-Assembling Polydopamine Microcapsules for Biomedical Applications

Bioadhesive microporous architectures that mimic the functions of a natural extracellular matrix (ECM) were prepared by self-assembling polydopamine (PDA) microcapsules, which not only favor cell adhesion and growth, but also facilitate growth factor immobilization and release. PDA-coated polystyrene (PS) microspheres are synthesized by polymerization of dopamine on sulfonated PS microspheres and then assembled using positively charged chitosan (CHI) layers as link agents. After the PS core templates were removed, microporous architectures composed of PDA microcapsules were obtained. The produced microporous PDA architectures have a high capability of adsorbing BMP-2 and realize the sustained release of BMP-2. More importantly, the bioadhesive micro architecture and its immobilized BMP-2 synergistically enhance the activity and osteogenetic differentiation of bone marrow mesenchymal stem cells (BMSCs). Both supercell adhesion and BMP-2 immobilization ability of these architectures are attributed to the intrinsic adhesive nature of PDA and the porous architectures via the assembly of PDA microcapsules. The bioadhesive microporous PDA architectures with both cell affinitive and GF release features have a great potential to mimic natural ECM for modifying various medical devices in the fields of tissue engineering and regenerative medicine